unbreak CI - #101

Describe your changes

CI on main was red -- 11 components failing. This fixes all of them bar two that turned out not to be ours. Each fix is its own commit, and everything here was verified locally before committing.

Wrong results, not just a red tick

• methods/harmonypy -- harmonypy 2.0 returns Z_corr as cells x pcs, where it used to be the other way around, so our transpose was handing anndata a (50, 600) array for a 600 cell dataset.

Metrics

• metrics/kbet_pg + metrics/kbet_pg_label -- pandas<2.0 and numpy<2.0 no longer resolve to anything that builds, and pegasuspy doesn't ask for them anyway (it wants pandas>=1.2.0 and declares zarr<3 itself). Dropping all three bounds gets the image building; pegasus then trips over pkg_resources, which setuptools dropped in 81, so that needs setuptools<81. These two being down was the worst of it -- a broken metric leaves a hole in every method's row.
• metrics/bras -- base image :1.0.0 -> :1, see below.

Stale bounds that had drifted into contradiction

• methods/scalex -- numpy<1.24 and torch<2.1 contradict what scalex 1.0.4 asks for, so pip fell back to building an ancient numpy from source. Dropping them fixes the build, but scalex still calls np.Inf, so it does need numpy<2.
• methods/pyliger -- pulls in louvain, which has no wheel for python 3.12 and builds igraph's C core from source. Needs cmake.
• methods/stacas -- STACAS 2.3.0 calls GetAssayData(slot = ), defunct since SeuratObject 5.0.0. Fixed upstream in carmonalab/STACAS@9a85ec4c, but no release since May 2025, so this tracks master.

Base image :1.0.0 -> :1

cellplm, condo, drvi and bras were the only components left on :1.0.0, whose openproblems predates openproblems.project.check_config. Since #99 landed the common bump, the component tests import that, so all four died with ImportError: cannot import name 'check_config'. Bumping them exposed three real failures underneath:

• methods/cellplm -- same louvain/cmake problem as pyliger, with a twist: the pytorch base image ships a /usr/local/bin/cmake that is a python shim (from cmake import cmake). /usr/bin/python3 can't see the pip-installed cmake module, so the shim always dies and it shadows anything we apt install. Install cmake from apt and delete the shim. Worth knowing about for anything else on that base.
• methods/drvi -- drvi-py==0.1.7 pins scvi-tools 1.0.4, dragging numpy back to 1.26 while the base image's zarr 3.3 wants numpy>=2, so it died in zarr on numpy.dtypes.StringDType. Pinning zarr<3 instead only moved the problem to torch 2.3.1 missing torch.library.register_fake, so this bumps drvi-py to 0.2.7. This changes drvi's results -- two minor versions.
• methods/condo -- fine once rebuilt on :1.

The two GPU foundation models

• methods/geneformer -- pip clones git+ urls with --filter=blob:none, and huggingface's git server no longer serves that (fatal: expected 'packfile'). Clone it ourselves instead; it's a 3.6 MB clone, the weights come from hf_hub_download at runtime. That exposed an ImportError on SpecialTokensMixin, which transformers dropped in 5 while the base image ships 5.15, hence transformers<5.
• methods/scgpt_zeroshot + methods/scgpt_finetuned -- the flash-attn install has never worked, and it turns out we don't need it: both scripts pass use_fast_transformer=False. Dropping it uncovered three more pins with no python 3.12 wheels -- numpy<1.24 -> numpy<2, torchtext==0.17.0 -> 0.17.2, and transformers==4.33.2 -> 4.36.2 (4.33 wants tokenizers<0.14, which needs a rust toolchain to build). Both images now build and import scgpt works on torch 2.2.2+cu121. Left torch, scvi-tools, datasets and cell-gears alone, they don't block anything. Shout if you'd rather keep transformers==4.33.2 and install rust instead -- heavier image, but bit-for-bit the old environment.

Left alone

• methods/limma_removebatcheffect and methods/liger -- both died because r2u.stat.illinois.edu timed out, and options(warn = 2) turns that apt warning into a fatal error. Nothing wrong on our side, and both pass on the re-run here.

A note on the GPU methods

cellplm, geneformer and both scgpt components merge base_method.yaml, whose only test resource is check_config.py. CI never runs their scripts, so a green tick here means the image builds and the config is valid, nothing more. I verified import geneformer and import scgpt inside the built images by hand, but whether they still produce correct output isn't covered by anything we run today. Probably worth a separate look on de.NBI.

On pinning

I've taken bounds off wherever the package doesn't actually need them, so we get updates and hear about it when something breaks, and only kept the ones that are genuinely required: numpy<2 for scalex (np.Inf), setuptools<81 for pegasus (pkg_resources), transformers<5 for geneformer (SpecialTokensMixin).

Checklist before requesting a review

• I have performed a self-review of my code

• Check the correct box. Does this PR contain:

  • Breaking changes
  • New functionality
  • Major changes
  • Minor changes
  • Bug fixes

• Proposed changes are described in the CHANGELOG.md

• CI Tests succeed and look good!